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I. REAL PARTY IN INTEREST 

The real party in interest of Application No. 10/428,973 is: 

THOMSON LICENSING 
46, Quai A. Le Gallo 
F-92100 Boulogne-Billancourt 
France. 
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II. RELATED APPEALS AND INTERFERENCES 

There are no related Appeals or Interferences. 
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III. STATUS OF THE CLAIMS 

Claims 1-6 are pending in this application. 
Claims 1, 2, 3 and 6 have been rejected. 

Claims 4 and 5 are objected to as being dependent upon a rejected base claim. 
The rejections of claims 1, 2, 3 and 6 are appealed. 
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IV. STATUS OF AMENDMENTS 

In response to the FINAL Office Action dated April 18, 2008, Applicants' 
representative filed a response traversing the rejection with no amendments to the claims. In 
the Advisory Action dated July 1, 2008, the Examiner maintained the rejection. 

As such, this appeal is directed to the claims as they stood at the time of the 
Advisory Action dated July 1, 2008, v^hich are shov^n in the Claims Appendix of this Brief. 
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V. SUMMARY OF CLAIMED SUBJECT MATTER 

Applicants' claimed invention is directed to a method of clustering images of a video 
sequence. There is one independent claim pending in the application: claim 1 . 

Applicants' independent claim 1 is directed to a method of clustering images of a 
video sequence consisting of shots and represented by a graph-like structure, a node of the 
graph representing a shot or a class of shots defined by key images and the nodes being 
connected by edges. (Applicants' specification, p. 6, Ins. 9-10.) A first step is the selection 
of an edge ak connecting tv^o nodes ni and np (Applicants' specification, p. 12, Ins. 9-15; 
FIG. 3.) 

The next tv^o steps are the calculation of the potential of a node nm, foUov^ed by the 
merging of the tv^o nodes ni and nj, as a function of the distances betv^een the attributes of 
the key images defining the class of shots of node ni and those of the key images defining 
the class of shots of node nj and as a function of the temporal distance of these key images. 
(Applicants' specification, p. 12, Ins. 9 - 15 and 17 - 19; p. 9, Ins. 7 - 12; and p. 9, In. 13; 
FIG. 3.) 

The next step is the calculation of the potential of each edge connecting the merged 
node to another node of the graph previously connected to nodes ni or nj, as a function of the 
distances between the attributes of the key images defining the class of shots of the merged 
node and those of the key images defining the class of shots of the other node and as a 
function of the temporal distance betv^een these key images, the new class of shots 
associated with the merged node comprising the key images of the classes of shots of the 
merged nodes. (Applicants' specification, p. 7, Ins. 19-28; p. 12, Ins. 19-23; p. 6, Ins. 35-39; 
FIG. 3.) 

The final step is the merging of the two nodes and validation of the new graph if the 
energy of this graph, which is the sum of the potentials of the nodes and of the edges, is less 
than the energy of the graph before merging. (Applicants' specification, p. 12, Ins. 23 -31; 
FIG. 3.) 
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VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

There is one ground of rejection to be reviewed on Appeal. 

(1) Whether claims 1, 2, 3 and 6 are unpatentable under 35 U.S.C. § 103(a) as being 
obvious over U.S. Patent No. 5,821,945 issued October 13, 1998 to Yeo et al. (Yeo) in view 
of U.S. Patent No. 7,054,367 issued May 30, 2006 to Oguz et al. (Oguz) and further in view 
of U.S. Patent No. 7,212,201 issued May 1, 2007 to Geiger et al. (Geiger). 
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VII. ARGUMENT 

Rejection of Claims 1, 2, 3 and 6 under 35 U.S.C. §103(a) as being unpatentable over 
Yeo in view of Oguz and further in view of Geiger 

CLAIMS L 2. 3 and 6 

Claim 1 is patentable over Yeo in view of Oguz and further in view of Geiger for 
die simple reason diat die combination of diese references does not yield Applicants' 
claimed invention. 

The Examiner states that "Oguz describes temporal distance'' as claimed by 
Applicants, i.e., that Oguz describes "[k]ey frames or features/attributes of a video 
sequence change - col. 5, line 39 to col. 6, line 35". Applicants respectfully submit that the 
Examiner is wrong and Oguz does not describe temporal distance as claimed by 
Applicants. For example, in this portion of Oguz, the values of the DC coefficients of the 
DCT blocks, which are differentially coded (except for the first one of die slice), are 
compared, for neighboring blocks, in order to detect an edge. The lengdis diat are 
compared to a direshold are die lengdis of die VLC coded words coding die DC 
coefficients of die blocks of an image. In point of fact, in diis portion of Oguz processing 
is only being performed on an image and diere is no mention of a temporal distance 
relating to images as required by Applicants' claim 1. 

Applicants also note that die Examiner states diat Oguz describes temporal distance 
by computing "a degree of coincidence between significant edges in a current frame and 
significant edges in a prior frame to within a distance and temporal distance" on col. 8, 2"^ 
paragraph of Oguz. Again, with respect to the requirements of Applicants' claim 1, the 
Examiner is wrong. This distance is only a spatial distance . 

Applicants do note that the distance disclosed on line 14 of Oguz is a temporal 
distance between the current and prior frames. But, this temporal distance is used to 
determine the amount of motion in the scene (Oguz, line 15). The motion amplitude is 
calculated between a first frame corresponding to a first instant and a second frame 
corresponding to a second instant and this depends, for a given speed of the object moving 
within the scene, on the temporal distance between die two instants. This temporal 
distance allows one to determine the number of blocks to consider (vicinity) in order to 
calculate the similarities (the degree of coincidence between significant edges in a prior 
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frame mentioned in line 9). Consequently, the disclosed temporal distance in Oguz - which 
is the one from the current frame to previous frames - is used to scale motion in order to 
define vicinity and to compute a coincidence coefficient in order to detect a scene cut, as 
explained in lines 13-15 of Oguz. 

In contrast. Applicants' claimed invention is different. In Applicants' claimed 
invention temporal distance is not used to define criteria (such as vicinity) to perform 
correlation calculations as in Oguz or to calculate attribute differences. The way the 
attribute differences are calculated is not specified in claim 1. What Applicants do claim 
and disclose is the calculation of a potential of an edge according to the temporal distance 
between the key images relating to the nodes connecting this edge. What Applicants also 
claim is the decision of merging two nodes depending on the temporal distance of the key 
images relating to these nodes. In Applicants' claimed invention, the potential of the edge 
linking two key frames is a function of the temporal distance. 

In contrast to Applicants' claimed invention, in Oguz the temporal distance is only 
used to scale motion to define vicinity for the calculation of the coincidence coefficients. 
One skilled in the art knows that the size of correlation search windows depend on the 
temporal distance between frames as the motion amplitude increases with the temporal 
distance. But - the value of the coincidence coefficient is not a function of the temporal 
distance. For example, a current frame identical to frame Fl and frame F2 and 
respectively temporally distant of Dl and D2 gives a same coincidence coefficient in Oguz 
whatever Dl and D2. However in Applicants' claimed invention, the potential of the edge 
linking the current frame to Fl will be different from the potential of the edge linking the 
current frame to F2 if Dl is different from D2. 

As explained in Applicants' specification (p 13, lines 34-39), the method allows to 
merge similar images even if they are distant (temporally speaking) or less similar images 
if they are close. The advantage is that it is possible to take into account, for the merging, 
scenes which are very far from each other, due to the iterations making key images 
becoming closer within the graph (page 4, lines 7-11), without increasing calculations (no 
need to limit the number of frames or shots to limit calculations as in prior art, top of page 
2 of the specification). As explained on page 4, lines 21-24 of Applicants' specification, 
the advantage is the taking into account of images even if they are temporally far apart 
while favoring the clustering of temporally close images. In a different way, it could be 
said that there are two steps in the process. The first step is the calculation of the 
difference in attributes, or the coincidence coefficient between two frames. The second 
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Step is the calculation of the potential of an edge. Oguz relates to the first step, Applicants' 
claim the second step. 

In summary, none of the cited prior art describes or suggests Applicants' claimed 
invention. 

Yeo describes a method for video browsing based on content and structure. A 
hierarchical scene transition graph is used. The nodes of the graph capture the core 
contents of the video while the edges capture its structure. At each level of the hierarchy, a 
different criterion for clustering is imposed. The clustering algorithms are built on 
dissimilarity measures defined for color, shape. In Yeo there is no description or 
suggestion about temporal distances and merging of nodes according to potentials which 
are function of these temporal distances as claimed by Applicants. There is no description 
or suggestion in Yeo about using the potential of the edges for deciding the merging of 
nodes as claimed by Applicants. 

Oguz describes a process to detect a scene change by considering significant edges 
in a current frame and significant edges in a previous frame. The temporal distance 
between the current frame and the prior frames and the amount of motion are taken into 
account to define the region where the coincidence coefficient is computed. Oguz 
describes the use of a temporal distance to determine parameters (number of blocks, see 
line 13-15 of col. 6) in order to calculate similarities between frames for the detection of 
the cut. Oguz does not describe or suggest about a use of the temporal distance between 
two frames to calculate the potential of the edge linking these two frames as claimed by 
Applicants. 

In contrast to the cited art. Applicants claim 1 specifies that the potential of an edge 
depends on the distance between the attributes of the nodes at the end of the edge and on 
the temporal distance between these nodes. The similarity parameter (distance between 
attributes) is combined to the temporal distance parameter for the decision of merging, 
giving a more reliable clustering as distant frames are taken into account while weighting is 
applied. This is not described or suggested in any combination of the cited prior art. 

As explained in Applicants' specification, prior art discloses that, if two shots are 
separated by more than T images, the clustering is not possible (see page 2, lines 17-20). T 
has a fixed value. T is not matched to the sequence leading, for example, to overchopping. 
As such, the use of a temporal distance to calculate a similarity between frames, such as the 
one described in Oguz (temporal distance relating to close frames for scaling motion), is 
very common as the motion amplitude depends on this distance. Nevertheless, Yeo, which 
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describes a graph structure, doesn't suggest such a use of the temporal distance for the 
calculation of the graph structure. 

In view of the above, there is no description or suggestion Yeo, Oguz or Geiger, 
either singly or in any combination, about a combination of parameters such as a temporal 
distance and a distance between attributes to decide the merging of nodes as required by 
Applicants' claim 1. 

Applicants' dependent claims 2, 3 and 6 stand or fall with Applicants' independent 
claim 1. 

In view of the above, Applicants' independent claim 1, and, therefore, claims 2, 3 
and 6 are patentable over (Yeo) in view Oguz and further in view of Geiger, 

VIIL CONCLUSION 

For the above reasons. Applicants submit that claims 1, 2, 3 and 6 are patentable. It 
is therefore respectfully requested that 

- the rejection of claims 1, 2, 3 and 6 under 35 U.S.C. § 103(a); 
be reversed. 

Respectfully submitted, 
Lionel Oisel et al. 

B v /Joseph J. Opalach/ 

Joseph J. Opalach 

Registration No.: 36,229 
(609) 734-6839 

Patent Operations 
Thomson Licensing LLC. 
P.O. Box 5312 

Princeton, New Jersey 08543-5312 
November 17, 2008 
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IX, CLAIMS APPENDIX 

1. (Previously presented) Method of clustering images of a video sequence 
consisting of shots and represented by a graph-like structure, a node of the graph 
representing a shot or a class of shots defined by key images and the nodes being connected 
by edges, comprising the following iteration: 

selecting an edge a^ connecting nodes ni and nj 
calculating a potential of node nm, 

merging of the two nodes ni and nj, as a function of the distances between the 
attributes of the key images defining the class of shots of node ni and those of the key 
images defining the class of shots of node nj and as a function of the temporal distance of 
these key images, 

calculating a potential of each edge connecting the merged node to another node of 
the graph previously connected to nodes ni or nj, as a function of the distances between the 
attributes of the key images defining the class of shots of the merged node and those of the 
key images defining the class of shots of the other node and as a function of the temporal 
distance between these key images, the new class of shots associated with the merged node 
comprising the key images of the classes of shots of the merged nodes, and 

merging of the two nodes and validation of the new graph if an energy of this graph, 
which is the sum of the potentials of the nodes and of the edges, is less than the energy of 
the graph before merging. 

2. (Previously presented) Method according to Claim 1, wherein the graph is 
initialized by assigning a node to each shot and in that edges are created from one node to 
another node if the shots relating to these nodes are separated by a predetermined maximum 
number T of shots. 

3. (Previously presented) Method according to Claim 1, wherein the value of 
potential associated with the edges and with the nodes is a respectively decreasing and 
increasing function of temporal distance. 



13 



CUSTOMER NO. 24498 
Application No.: 10/877,581 



PF020128 



4. (Previously presented) Method according to claim 2, wherein the potential of an 
edge connecting two nodes ni and nj is calculated according to the following equation: 

K ( ' ) = , min {w^„ ' (K^,, , d^,, , d^^ (I, J,)) + Wj - F^(K^J^, d^ (I, , ))) 

and in that the potential of a node bom from the merging of two nodes ni and nj is 
defined by the following function: 

V„ (n^ =n,[jnj)= min ( w,, • F„ (K^^ , d^, , d,, (/, ,/,)) + • F„ (i^T, , , (/, , I, ))) 
where: 

Pi represents the class of shots associated with node rii and comprising images 4 
Fa is a decreasing function 
F„ is an increasing function 

Watt and wr represent the weights related to attributes and to time 
dattihji) is the distance function associated with the characteristic attributes of the 
two images 

d^tt represents the mean of the distances between images calculated on the initial 

graph 

djihyJi) is the temporal distance separating the two images 
dx is equal to T which represents the temporal threshold 
Katt and Kt are two constants. 



5. (Previously presented) Method according to claim 4, wherein the decreasing 
function is of the form 

1 



Fa(K,d,d) = 



^(d-d) 
1 + ed 



and in that the increasing function is of the form 
1 



Fn(K,d,d) = l- 



^(d-d) 
1 + ed 
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6. (Previously presented) Method according to claim 1, wherein the iterations are 
stopped as soon as the potential merging of two nodes gives rise to an increase in the 
energy, the edges being selected, for the calculation of the mergings, in decreasing order of 
their value of potential. 
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X. EVIDENCE APPENDIX (NONE) 
None. 
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XI, RELATED PROCEEDINGS APPENDIX (NONE) 

None. 



